{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/obss/BIOBSS/blob/main/examples/acc_processing.ipynb)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__BIOBSS - ACC Signal Processing__\n",
    "\n",
    "_This notebook includes guidelines to help using BIOBSS for calculation of activity metrics and feature extraction from ACC signals._"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__To run this notebook in Colab:__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "git clone https://github.com/obss/biobss.git\n",
    "cd biobss\n",
    "pip install ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U matplotlib\n",
    "import shutil\n",
    "from biobss import FIXTURES_ROOT\n",
    "shutil.move(\"/content/biobss/sample_data\",FIXTURES_ROOT.parent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Import required packages:__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import biobss\n",
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Table of Contents\n",
    "1. [ACC Sample Data](#sampledata)<br>\n",
    "2. [ACC Signal Preprocessing](#acc_pre)<br>\n",
    "    2.1. [Filtering](#acc_filter)<br>\n",
    "    2.2. [Peak Detection](#acc_peaks)<br>\n",
    "    2.3. [Plotting](#acc_plot)<br>\n",
    "3. [Activity Metrics from ACC Signals](#acc_actind)<br>\n",
    "    3.1. [Dataset Generation](#acc_dataset)<br>\n",
    "    3.2. [Calculation of Activity Metrics](#act_ind)<br>\n",
    "4. [ACC Feature Extraction](#acc_features)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __ACC Sample Data__\n",
    "<a id=\"sampledata\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ACC sample data is provided as a csv file in BIOBSS\\sample data. The data file contains 3-axis ACC signals of 5-minutes length, sampled at 32 Hz. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Load the sample data\n",
    "data, info = biobss.utils.load_sample_data(data_type='ACC')\n",
    "accx = np.asarray(data['ACCx'])\n",
    "accy = np.asarray(data['ACCy'])\n",
    "accz = np.asarray(data['ACCz'])\n",
    "fs = info['sampling_rate']\n",
    "L = info['signal_length']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __ACC Signal Preprocessing__\n",
    "<a id=\"acc_pre\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Filtering__\n",
    "<a id=\"acc_filter\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BIOBSS provides the __filter_signal__ function which filters signals with Butterworth filter designed using __Scipy__. The filter parameters (filter type, filter order, cutoff frequencies) should be defined as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Filtering ACC signals by defining the filter parameters\n",
    "\n",
    "f_accx= biobss.preprocess.filter_signal(sig=accx, sampling_rate=fs, filter_type='lowpass', N=2, f_upper=10)\n",
    "f_accy= biobss.preprocess.filter_signal(sig=accy, sampling_rate=fs, filter_type='lowpass', N=2, f_upper=10)\n",
    "f_accz= biobss.preprocess.filter_signal(sig=accz, sampling_rate=fs, filter_type='lowpass', N=2, f_upper=10)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an alternative, pre-defined filters can be used for each signal type. For this purpose, _signal_type_ should be 'ACC' and _method_ should be selected. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Filter ACC signal by using predefined filters\n",
    "filtered_accx=biobss.preprocess.filter_signal(sig=accx, sampling_rate=fs, signal_type='ACC', method='lowpass')\n",
    "filtered_accy=biobss.preprocess.filter_signal(sig=accy, sampling_rate=fs, signal_type='ACC', method='lowpass')\n",
    "filtered_accz=biobss.preprocess.filter_signal(sig=accz, sampling_rate=fs, signal_type='ACC', method='lowpass')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Peak Detection__\n",
    "<a id=\"acc_peaks\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ___peak_detection___ function can be used to detect peaks in the ACC signal(s). Method can be 'peakdet', 'heartpy' or 'scipy'. The function returns a dictionary including arrays of peak and trough locations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Detect peaks using 'peakdet' method (delta=0.01). \n",
    "#Delta parameter should be adjusted related to the amplitude of the signal.\n",
    "\n",
    "info=biobss.preprocess.peak_detection(sig=filtered_accx, sampling_rate=fs, method='peakdet', delta=0.01)\n",
    "\n",
    "locs_peaks=info['Peak_locs']\n",
    "peaks=filtered_accx[locs_peaks]\n",
    "locs_onsets=info['Trough_locs']\n",
    "onsets=filtered_accx[locs_onsets]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Plotting__\n",
    "<a id=\"acc_plot\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BIOBSS provides plotting functions specific to each signal type. In order to plot ACC signals, ___plot_acc___ function can be used. The _signals_ and _peaks_ should be dictionaries and the keys should be selected properly as shown below. The plots can be generated either using __Matplotlib__ or __Plotly__. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate inputs as dictionaries\n",
    "signals = {'x-axis':{'Raw': accx , 'Filtered': filtered_accx}, 'y-axis':{'Raw': accy , 'Filtered': filtered_accy}, 'z-axis':{'Raw': accz , 'Filtered': filtered_accz}}\n",
    "peaks = {'x-axis':{'Raw':{'Peaks': locs_peaks}, 'Filtered':{'Peaks': locs_peaks}}}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot ACC signals using Matplotlib (default)\n",
    "biobss.imutools.plot_acc(signals=signals, peaks=peaks, sampling_rate=fs, method='matplotlib', show_peaks=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot ACC signals using Plotly\n",
    "biobss.imutools.plot_acc(signals=signals, peaks=peaks, sampling_rate=fs, method='plotly', show_peaks=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __Activity Metrics from ACC Signals__\n",
    "<a id=\"acc_actind\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BIOBSS provides a set of functions to calculate activity metrics defined in the literature. These activity metrics are:\n",
    "\n",
    "- Proportional Integration Method (PIM)\n",
    "- Zero Crossing Method (ZCM)\n",
    "- Time Above Threshold (TAT)\n",
    "- Mean Amplitude Deviation (MAD)\n",
    "- Euclidian Norm Minus One (ENMO)\n",
    "- High-pass Filtered Euclidian (HFEN)\n",
    "- Activity Index (AI)\n",
    "\n",
    "Reference: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0261718\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Dataset Generation__\n",
    "<a id=\"acc_dataset\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The preprocessing steps which should be applied on the raw acceleration signal are different for each of the activity metrics listed above. In other words, each activity index can be calculated only from specific datasets. BIOBSS package has the ___generate_dataset___ function which applies appropriate preprocessing steps (dataset generation) to prevent errors and make the process easier for the users.\n",
    "\n",
    "The generated datasets are:\n",
    "- UFXYZ: unfiltered acc signals \n",
    "- UFM: magnitude of unfiltered acc signals \n",
    "- UFM_modified: modified magnitude of unfiltered signals (absolute(UFM-length(UFM)))\n",
    "- UFNM: normalized magnitude of unfiltered acc signals \n",
    "- FXYZ: filtered acc signals\n",
    "- FXYZ_modified: modified filtered acc signals (absolute(FXYZ))   \n",
    "- FMpre: magnitude of filtered acc signals\n",
    "- SpecialXYZ: filtered acc signals (special filter parameters)  \n",
    "- SpecialM: magnitude of filtered acc signals (special filter parameters)\n",
    "- FMpost: filtered magnitude of acc signals\n",
    "- FMpost_modified: modified of filtered magnitude of acc signals (absolute(FMpost))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset generation step is a part of activity index calculation pipeline and the ___calc_activity_index___ function can handle it by calling the ___generate_dataset___ function. However, this function can also be used independently by defining input arguments properly as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate the datasets\n",
    "ufxyz           = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=False,filtering_order=None,magnitude=False,normalize=False,modify=False),\n",
    "ufm             = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=False,filtering_order=None,magnitude=True,normalize=False,modify=False),\n",
    "ufm_modified    = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=False,filtering_order=None,magnitude=True,normalize=False,modify=True),\n",
    "ufnm            = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=False,filtering_order=None,magnitude=True,normalize=True,modify=False),\n",
    "fxyz            = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='pre',magnitude=False,normalize=False,modify=False),\n",
    "fxyz_modified   = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='pre',magnitude=False,normalize=False,modify=True),\n",
    "fmpre           = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='pre',magnitude=True,normalize=False,modify=False),\n",
    "specialxyz      = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='pre',magnitude=False,normalize=False,modify=False,filter_type='highpass',N=4,f_lower=0.2),\n",
    "specialm        = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='pre',magnitude=True,normalize=False,modify=False,filter_type='highpass',N=4,f_lower=0.2),\n",
    "fmpost          = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='post',magnitude=True,normalize=False,modify=False),\n",
    "fmpost_modified = biobss.imutools.generate_dataset(accx,accy,accz,fs,filtering=True,filtering_order='post',magnitude=True,normalize=False,modify=True)\n",
    "                    "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Calculation of Activity Metrics__\n",
    "<a id=\"act_ind\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ___calc_activity_index___ function is used to calculate an activity index defined by _metric_ for the selected _input_types_. Note that if the _input_types_ is not passed to the function, the activity metric is calculated for all of the valid input types.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculate activity metrics\n",
    "pim = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='PIM')\n",
    "zcm = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='ZCM')\n",
    "tat = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='TAT')\n",
    "mad = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='MAD')\n",
    "enmo = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='ENMO')\n",
    "hfen = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='HFEN')\n",
    "ai = biobss.imutools.calc_activity_index(accx, accy, accz, signal_length=60, sampling_rate=fs, metric='AI', baseline_variance=[0.5,0.5,0.5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __ACC Feature Extraction__\n",
    "<a id=\"acc_features\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The features which are used for analysis of ACC signals can be categorized as statistical features, frequency domain features and correlation features. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Statistical features:\n",
    "- mean: mean of the signal amplitude\n",
    "- std: standard deviation of the signal amplitude\n",
    "- mad: mean absolute deviation of the signal amplitude\n",
    "- min: minimum value of the signal amplitude\n",
    "- max: maximum value of the signal amplitude\n",
    "- range: difference of maximum and minimum values of the signal amplitude\n",
    "- median: median value of the signal amplitude\n",
    "- medad: median absolute deviation of the signal amplitude\n",
    "- iqr: interquartile range of the signal amplitude\n",
    "- ncount: number of negative values \n",
    "- pcount: number of positibe values \n",
    "- abmean: number of values above mean\n",
    "- npeaks: number of peaks\n",
    "- skew: skewness of the signal\n",
    "- kurtosis: kurtosis of the signal\n",
    "- energy: signal energy (the mean of sum of squares of the values in a window)\n",
    "- momentum: signal momentum\n",
    "\n",
    "Frequency domain features:\n",
    "- fft_mean: mean of fft peaks\n",
    "- fft_std: standard deviation of fft peaks\n",
    "- fft_mad: mean absolute deviation of fft peaks\n",
    "- fft_min: minimum value of fft peaks\n",
    "- fft_max: maximum value of fft peaks\n",
    "- fft_range: difference of maximum and minimum values of fft peaks\n",
    "- fft_median: median value of fft peaks\n",
    "- fft_medad: median absolute deviation of fft peaks\n",
    "- fft_iqr: interquartile range of fft peaks\n",
    "- fft_abmean: number of fft peaks above mean\n",
    "- fft_npeaks: number of fft peaks\n",
    "- fft_skew: skewness of fft peaks\n",
    "- fft_kurtosis: kurtosis of fft peaks\n",
    "- fft_energy: energy of fft peaks\n",
    "- fft_entropy: entropy of fft peaks\n",
    "- f1sc: signal power in the range of 0.1 to 0.2 Hz\n",
    "- f2sc: signal power in the range of 0.2 to 0.3 Hz\n",
    "- f3sc: signal power in the range of 0.3 to 0.4 Hz\n",
    "- max_freq: frequency of maximum fft peak\n",
    "\n",
    "Correlation features:\n",
    "- accx_accy_corr: correlation coefficient for x and y axes\n",
    "- accx_accz_corr: correlation coefficient for x and z axes\n",
    "- accy_accz_corr: correlation coefficient for y and z axes\n",
    "\n",
    "Reference: https://towardsdatascience.com/feature-engineering-on-time-series-data-transforming-signal-data-of-a-smartphone-accelerometer-for-72cbe34b8a60"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These features can be calculated seperately for each domain using the functions ___get_freq_features___, ___get_stat_features___ and ___get_corr_features___. The signal arrays and signal names should be lists for multi axis signals. Then, the features are calculated for each signal in the list. Note that list of _signals_ and _signal_names_ should have the same order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate list of signals and signal_names to be used in feature calculation.\n",
    "signals = [accx, accy, accz]\n",
    "signal_names = ['accx', 'accy', 'accz']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calculate frequency domain, statistical and correlation features for x-axis acceleration signal\n",
    "features_freq = biobss.imutools.acc_features.acc_freq_features(signals=signals, signal_names=signal_names, sampling_rate=fs)\n",
    "features_stat = biobss.imutools.acc_features.acc_stat_features(signals=signals, signal_names=signal_names, sampling_rate=fs)\n",
    "features_corr = biobss.imutools.acc_features.acc_corr_features(signals=signals, signal_names=signal_names, sampling_rate=fs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ___get_acc_features___ function is used to calculate features for multiple domains at a time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_acc = biobss.imutools.get_acc_features(signals=signals, signal_names=signal_names, sampling_rate=fs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to calculate the features for the signal vector magnitude ($\\sqrt{accx^2 + accy^2 + accz^2}$), _magnitude_ should be True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_acc = biobss.imutools.get_acc_features(signals, signal_names, sampling_rate=fs, magnitude=True)\n",
    "features_freq = biobss.imutools.acc_features.acc_freq_features(signals=signals, signal_names=signal_names, sampling_rate=fs, magnitude=True)\n",
    "features_stat = biobss.imutools.acc_features.acc_stat_features(signals=signals, signal_names=signal_names, sampling_rate=fs, magnitude=True)\n",
    "features_corr = biobss.imutools.acc_features.acc_corr_features(signals=signals, signal_names=signal_names, sampling_rate=fs)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "biolib",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "3a4bcfb23c7e6ad66c655087280fa9f4d0273121ae7909f7735d1e02563a2438"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
